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DETAILED ACTION 

This is a Non-Final Office Action on the merits. This action is responsive to 
RCE/Remarks, which was filed on 10/1 1/2007. 

Claims 1,4-5, 8-13, 16-17, 20-25, 28-29, 32-36, and 44-52, are pending. 
Claims 2-3, 6-7, 14-15, 18-19, 26-27, 30-31, and 37-43 are previously canceled. 
Claims 1,4-5, 8-9, 1, 13, 16-17, 20-21, 23, 25, 28-29, 32-33, 35, and 44-52 were 
previously presented. Claims 10, 12, 22, 24, 34, and 36 were original presented. 
Effective filing date is 03/15/2000, (Assignee: Ricoh). 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the 
fee set forth in 37 CFR 1 .17(e), was filed in this application after final rejection. 
Since this application is eligible for continued examination under 37 CFR 1.114, 
and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the 
previous Office action has been withdrawn pursuant to 37 CFR 1.114. 
Applicant's submission filed on 10/11/2007. 

Claim Objections 

Claim 45 objected to because of the following informalities: Claim 45 
recites the limitation "the method of claim 884" in Line 1; Claim 45 also recites 
the phrase "from a user +who speaks" in Line 2; the Examiner believes these are 
typo errors. Appropriate correction is required. 

In the interest of compact prosecution, the application is further examined 
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against the prior art, as stated below, upon the assumption that Claim 45 recites 
the limitation "the method of claim 44". 



Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

Claims 1, 4-5, 8-11, 13, 16-17, 20-23, 25, 28-29; 32-35, and 44-52 are 

rejected under 35 U.S.C. 103(a) as being as being unpatentable over Morris et 
al US005420974A issued 05/30/1995 (hereinafter Morris), in view of Schena et 
al US006448979B1 filed 01/25/1999 (hereinafter Schena). 

Regarding independent claim 1: Morris teaches: 

a method comprising: creating a multimedia annotation for a 
paper document, the multimedia annotation representing at least one 
of an audio and a video clip, 

(See Morris at Fig/ 9A-B and Col. 2, Lines 35-40, discloses scanned-in form, are 
stored in a mixed object document content architecture envelope (MODCA). 
Morris further discloses, if the box being defined is a check box, then the sub- 
type is defined as audio, video, image, text, or other type object. The resulting 
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directory and the image object of the overall scanned-in form are stored in a 
mixed object document content architecture envelope (MODCA)-see Morris at 
Col. 2 Lines 20-30.) 
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creating a first multimedia document by combining the paper 
document and the multimedia annotation represented by at least one 
of the audio sound and video clip, 

(See Morris at Col. 2, Lines 35-40, discloses scanned-in form, are stored in a 
mixed object document content architecture envelope (MODCA). Morris further 
discloses, if the box being defined is a check box, then the sub-type is defined as 
audio, video, image, text, or other type object. The resulting directory and the 
image object of the overall scanned-in form are stored in a mixed object 
document content architecture envelope (MODCA)-see Morris at Col. 2 Lines 20- 
30.) 

wherein the first multimedia document is generated as a part of 
reproducing the paper document via a document reproduction 
system, wherein the multimedia annotation is captured via an input 
device of the document reproduction system while the paper 
document is being reproduced via the document reproduction 
system, wherein the captured multimedia annotation is encoded, 
(See Morris at Col. 2, Lines 35-40, discloses scanned-in form, are stored in a 
mixed object document content architecture envelope (MODCA). Morris further 
discloses, if the box being defined is a check box, then the sub-type is defined as 
audio, video, image, text, or other type object. The resulting directory and the 
image object of the overall scanned-in form are stored in a mixed object 
document content architecture envelope (MODCA)-see Morris at Col. 2 Lines 20- 
30. 
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Also see Morris at Col. 2, Lines 25-40, discloses a filled-out hard copy of 
the form is desired to be entered into the image archiving system, the filled-out 
copy of the master form is scanned-in to the system. A special indicium is 
associated with each master form, which uniquely identifies that form and 
distinguishes it from all other master forms created for the system. The system 
reads the indicium from the scanned-in form and then accesses the MODCA 
envelope the directory for the master form is then accessed to obtain the 
coordinates of each component field on the scanned-in form. Then access the 
stored nonvisual object and play it back for the operator- See Morris at the 
Abstract.) 

wherein the first multimedia document, which when scanned by a 
process, the process causes the printed multimedia annotation to be 
decoded, the at least one of the audio sound and video clip to be 
extracted from the multimedia annotation, 

(See Morris at Col. 2, Lines 25-40, discloses a filled-out hard copy of the form is 
desired to be entered into the image archiving system, the filled-out copy of the 
master form is scanned-in to the system. A special indicium is associated with 
each master form, which uniquely identifies that form and distinguishes it from all 
other master forms created for the system. The system reads the indicium from 
the scanned-in form and then accesses the MODCA envelope the directory for 
the master form is then accessed to obtain the coordinates of each component 
field on the scanned-in form.) 
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and the at least one extracted audio sound and video clip can be 
played via a multimedia player. 

(See Morris at Col. 3, Lines 35-45, discloses when the check box is selected, the 
multimedia record is accessed from the MODCA envelope, for example, and is 
played. If it is an audio record, then the sound is played for the operator. If it is a 
video record, then a video cameo display window can be presented to the 
operator on the display screen and the video sequence played (i.e. played via a 
multimedia player). 

In addition, Morris does not teach, but Schena teaches: 

multimedia annotation represented by a first bar code, 
wherein the captured multimedia annotation is encoded within the 
first bar code, 

(See Schena at Col. 3 Lines 60-65, discloses high density barcodes. 

Also see Schena Col. 1 lines 10-25, discloses multimedia information (i.e. 
audio, video) using a scanner for machine-readable code containing a link 
information corresponding to a provider information depicted on the printed 
medium). 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify Morris's check box and the sub-type is defined as audio, 
video, image, text, or other type object scanned-in form, are stored in a mixed 
object document content architecture envelope (MODCA), to include a means of 
utilizing the barcode as taught by Schena, so that, when selected by a user (i.e. 
scanned document with the box is check as taught by Morris ) the barcode would 



, Application/Control Number: Page 8 

09/526,031 
Art Unit: 2176 

accessing a segment of video file which references scanned document, and 
provides a predictable result of the bridging the gap between the virtual 
multimedia-based of the Internet word and the physical world of printed media, 
wherein the annotate media may contain one or more of textual, audio, or video 
information, and the customer premises equipment ("CPE"), serving as the 
receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 
through col. 2. line 40). 

Regarding independent claim 13, 

Is directed to machine-readable medium providing instructions, 
which when executed by a set of one or more processors to perform the 
method of claim 1 which cites above, and is similarly rejected under the 
same rationale, Also See Morris at Fig. 1 and also at Col. 5, Lines 10-20. 

Regarding independent claim 25, 

Is directed to a computer system include a bus, a data storage, and 
a processor to perform the method of claim 1 which cites above, and is 
similarly rejected under the same rationale, Also See Morris at Fig. 1 and 
also at Col. 5, Lines 10-20. 
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Regarding claims 4, 16, and 28, Morris teaches: 

wherein a location indicator associated with the multimedia 
annotation is placed on the first multimedia document, wherein the 
location indicator indicates where the multimedia annotation can be 
retrieved and played. 
(See Morris at Col. 3, Lines 35-45, discloses when the check box is selected, the 
multimedia record is accessed from the MODCA envelope, for example, and is 
played. If it is an audio record, then the sound is played for the operator. If it is a 
video record, then a video cameo display window can be presented to the 
operator on the display screen and the video sequence played (i.e. played via a 
multimedia player). 

Regarding claims 5, 17, and 29, 

Morris does not expressly teach, but Schena teaches: 

wherein the location indicator comprises a first Uniform 

Resource Locator (URL), and a second bar code, wherein the first 

URL is indicated in plain text, and wherein the second bar code 

represents the first URL in an encrypted form. 
(Schena col. 4, lines 45-55, discloses the link information corresponding to the 
provider information may include a universal resource locator (URL) (Schena 
col.1, line 64 through col. 2. line 40). In addition, the links can be encoded 
according to provider, for example UPC or ISBN numbers and any code may 
serve as the alphanumeric sequence. 
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Also See Schena at Col. 3 Lines 60-65, discloses high density barcodes. 
Under the broadest reasonable interpretation, The Examiner equates the claimed 
bar code as equivalent to the UPC or ISBN numbers and any code as taught by 
Schena. This interpretation is supported by Applicant's Specification, which 
states "The annotation may be in different forms, such as, for example, a bar 
code containing an audio message or a URL indicating a link to a video clip" at 
page 7 lines 15-18. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify schena's a user selectable object (i.e. the location 
indicator comprises a first Uniform Resource Locator (URL), and a second bar 
code, wherein the first URL is indicated in plain text, and wherein the second bar 
code represents the first URL in an encrypted form) providing a user with access 
to the portion of the recorded information corresponding to the visual feature (i.e. 
machine readable barcode), to includes a means of selecting by a user onto a 
check box, the sub-type is defined as audio, video, image, text, or other type 
object scanned-in form, are stored in a mixed object document content 
architecture envelope (MODCA) as taught by Morris. One of ordinary skill in the 
art would have been motivated to modify this combination because Schena and 
Morris are from the same field of endeavor of electronic presentation includes 
annotation to presentation material, and provides a predictable result of the 
bridging the gap between the virtual multimedia-based of the Internet word and 
the physical world of printed media, wherein the annotate media may contain one 
or more of textual, audio, or video information, and the customer premises 
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equipment ("CPE"), serving as the receiver, plays multimedia sequence (Schena 
col.1, lines 5-15, col. 1 line 64 through col. 2. line 40). 

Regarding claims 8, 20, and 32, Morris teaches: 

creating a second multimedia document by combining the 
image of the paper document and the multimedia annotation; and 
storing the image of the paper document and the multimedia 
annotation in a storage, wherein the second multimedia document is 
an electronic document associated with the first multimedia 
document which is a physical document. 
(See Morris at Col. 2, Lines 35-40, discloses scanned-in form, are stored in a 
mixed object document content architecture envelope (MODCA). Morris further 
discloses, if the box being defined is a check box, then the sub-type is defined as 
audio, video, image, text, or other type object. The resulting directory and the 
image object of the overall scanned-in form are stored in a mixed object 
document content architecture envelope (MODCA)-see Morris at Col. 2 Lines 20- 
30. 

Also see Morris at Col. 2, Lines 25-40, discloses a filled-out hard copy of 
the form is desired to be entered into the image archiving system, the filled-out 
copy of the master form is scanned-in to the system. A special indicium is 
associated with each master form, which uniquely identifies that form and 
distinguishes it from all other master forms created for the system. The system 
reads the indicium from the scanned-in form and then accesses the MODCA 
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envelope the directory for the master form is then accessed to obtain the 
coordinates of each component field on the scanned-in form. Then access the 
stored nonvisual object and play it back for the operator- See Morris at the 
Abstract.) 

generating an image of the paper document, the image of the 
paper document being unconsciously captured via the document 
reproduction system during the reproduction of the paper document 
without user intervention; 

(See Morris at Col. 2, Lines 25-40, discloses a filled-out hard copy of the form is 
desired to be entered into the image archiving system, the filled-out copy of the 
master form is scanned-in to the system. A special indicium is associated with 
each master form, which uniquely identifies that form and distinguishes it from all 
other master forms created for the system. The system reads the indicium from 
the scanned-in form and then accesses the MODCA envelope the directory for 
the master form is then accessed to obtain the coordinates of each component 
field on the scanned-in form. Then access the stored nonvisual object and play it 
back for the operator- See Morris at the Abstract.) 

Regarding claims 9, 21, and 33, 

Morris does not expressly teach, but Schena teaches: 

wherein a multimedia document is represented as a second 
Uniform Resource Locator (URL) printed on the first multimedia 
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document, and wherein the image of the paper document and the 
multimedia annotation is accessed with the second URL, 

(Schena col. 4, lines 45-55, discloses the link information corresponding to the 
provider information may include a universal resource locator (URL) (Schena 
col.1, line 64 through col. 2. line 40). In addition, the links can be encoded 
according to provider, for example UPC or ISBN numbers and any code may 
serve as the alphanumeric sequence. 

Also See Schena at Col. 3 Lines 60-65, discloses high density barcodes. 
Under the broadest reasonable interpretation, The Examiner equates the claimed 
bar code as equivalent to the UPC or ISBN numbers and any code as taught by 
Schena. This interpretation is supported by Applicant's Specification, which 
states "The annotation may be in different forms, such as, for example, a bar 
code containing an audio message or a URL indicating a link to a video clip" at 
page 7 lines 15-18.) 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify schena's a user selectable object (i.e. the location 
indicator comprises a first Uniform Resource Locator (URL), and a second bar 
code, wherein the first URL is indicated in plain text, and wherein the second bar 
code represents the first URL in an encrypted form) providing a user with access 
to the portion of the recorded information corresponding to the visual feature (i.e. 
machine readable barcode), so that, when selected by a user the barcode would 
accessing a segment of video file which references scanned document as taught 
by Morris. One of ordinary skill in the art would have been motivated to modify 
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this combination because Schena and Morris are from the same field of 
endeavor of electronic presentation includes annotation to presentation material, 
and provides a predictable result of the bridging the gap between the virtual 
multimedia-based of the Internet word and the physical world of printed media, 
wherein the annotate media may contain one or more of textual, audio, or video 
information, and the customer premises equipment ("CPE"), serving as the 
receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 

Regarding claims 10, 22, and 34, 

Morris does not expressly teach, but Schena teaches: 

wherein a third bar code is used to represent a second URL 

(Schena col. 4, lines 45-55, discloses the link information corresponding to the 
provider information may include a universal resource locator (URL) (Schena 
col.1 , line 64 through col. 2. line 40). In addition, the links can be encoded 
according to provider, for example UPC or ISBN numbers and any code may 
serve as the alphanumeric sequence. 

Also See Schena at Col. 3 Lines 60-65, discloses high density barcodes. 
Under the broadest reasonable interpretation, The Examiner equates the claimed 
bar code as equivalent to the UPC or ISBN numbers and any code as taught by 
Schena. This interpretation is supported by Applicant's Specification, which 
states "The annotation may be in different forms, such as t for example, a bar 
code containing an audio message or a URL indicating a link to a video clip" at 
page 7 lines 15-18.) 
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It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify schena's a user selectable object (i.e. the location 
indicator comprises a first Uniform Resource Locator (URL), and a second bar 
code, wherein the first URL is indicated in plain text, and wherein the second bar 
code represents the first URL in an encrypted form) providing a user with access 
to the portion of the recorded information corresponding to the visual feature (i.e. 
machine readable barcode), so that, when selected by a user the barcode would 
accessing a segment of video file which references scanned document as taught 
by Morris. One of ordinary skill in the art would have been motivated to modify 
this combination because Schena and Morris are from the same field of 
endeavor of electronic presentation includes annotation to presentation material, 
and provides a predictable result of the bridging the gap between the virtual 
multimedia-based of the Internet word and the physical world of printed media, 
wherein the annotate media may contain one or more of textual, audio, or video 
information, and the customer premises equipment ("CPE"), serving as the 
receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 

Regarding claims 11, 23, and 35, Morris teaches: 

wherein the recipient is specified by a user via an interface of 
the document reproduction system when the user reproduces the 
paper document using the document reproduction system. 

(See Morris at Col. 2, Lines 35-40, discloses scanned-in form, are stored in a 
mixed object document content architecture envelope (MODCA). Morris further 
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discloses, if the box being defined is a check box, then the sub-type is defined as 
audio, video, image, text, or other type object. The resulting directory and the 
image object of the overall scanned-in form are stored in a mixed object 
document content architecture envelope (MODCA)-see Morris at Col. 2 Lines 20- 
30.) 

In addition, Morris does not expressly teach, but Schena teaches: 

automatically sending a second multimedia document to a 
recipient by electronic mail as a part of reproducing the paper 
document via the document reproduction system, 

(See Schena at Col. 4 lines 35, discloses the link information was published or 
located, along with message-specific information. The examiner equates the 
claimed automatically sending a second multimedia document to a recipient by 
electronic mail as equivalent to link information was published or located, along 
with a message as taught by Schena.) 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify schena's automatically sending a second multimedia 
document to a recipient by electronic mail as a part of reproducing the paper 
document, so that, a user can accessing a segment of video file which 
references scanned document as taught by Morris. One of ordinary skill in the art 
would have been motivated to modify this combination because Schena and 
Morris are from the same field of endeavor of electronic presentation includes 
annotation to presentation material, and provides a predictable result of the 
bridging the gap between the virtual multimedia-based of the Internet word and 
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the physical world of printed media, wherein the annotate media may contain one 
or more of textual, audio, or video information, and the customer premises 
equipment ("CPE"), serving as the receiver, plays multimedia sequence (Schena 
col.1, lines 5-15, col. 1 line 64 through col. 2. line 40). 

Regarding claim 44, Morris teaches: 

wherein the first multimedia document is a physical document 
having the first check box printed thereon, which when scanned by a 
scanning device, causes the first check box to be decoded and the 
audio sound to be extracted from the first check box, and wherein 
the extracted audio sound is capable of being played via an audio 
player. 

(See Morris at Col. 3, Lines 35-45, discloses when the check box is selected, the 
multimedia record is accessed from the MODCA envelope, for example, and is 
played. If it is an audio record, then the sound is played for the operator. If it is a 
video record, then a video cameo display window can be presented to the 
operator on the display screen and the video sequence played (i.e. played via a 
multimedia player). 

In addition, Morris does not expressly teach, but Schena teaches: 
bar code, 

(See Schena at Col. 3 Lines 60-65, discloses high density barcodes.) 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify Morris's check box and the sub-type is defined as audio, 
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video, image, text, or other type object scanned-in form, are stored in a mixed 
object document content architecture envelope (MODCA), to include a means of 
utilizing the barcode as taught by Schena, so that, when selected by a user (i.e. 
scanned document with the box is check as taught by Morris ) the barcode would 
accessing a segment of video file which references scanned document, and 
provides a predictable result of the bridging the gap between the virtual 
multimedia-based of the Internet word and the physical world of printed media, 
wherein the annotate media may contain one or more of textual, audio, or video 
information, and the customer premises equipment ("CPE"), serving as the 
receiver, plays multimedia sequence (Schena col.1, lines 5-15, col. 1 line 64 
through col. 2. line 40). 

Regarding claim 45, Morris teaches: 

capturing an audio sound of the multimedia annotation from a 
user using a microphone of the input device to annotate the paper 
document to create a multimedia paper document. 

(See Morris at Col. 2, Lines 5-20, discloses sound records, Video records, 
sequential animation sequence records, and the like, such multimedia objects 
being characterized in that their presentation requires duration of time. It is within 
the scope of the invention that check boxes can also be used to represent visual 
fields such as other image objects, other text objects, other vector graphics 
objects, etc. Morris at Col. 3 Lines 10-15, discloses the steps that allow a user to 
add new recorded segments of audio, video, etc. for any page of data. It is 
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inherently including the use of Microphone and Video camera in order to record 
sound and Video as taught by Morris.) 



Regarding claim 46, Morris teaches: 

wherein the microphone is automatically activated when the 
user selects a reproduction function of the document reproduction 
system to reproduce the paper document. 

(See Morris at Col. 2, Lines 17 -60, discloses sound records, Video records, 
sequential animation sequence records, and the like, such multimedia objects 
being characterized in that their presentation requires duration of time. It is within 
the scope of the invention that check boxes can also be used to represent visual 
fields such as other image objects, other text objects, other vector graphics 
objects, etc. Morris at Col. 3 Lines 10-15, discloses the steps that allow a user to 
add new recorded segments of audio, video, etc. for any page of data. It is 
inherently including the use of Microphone and Video camera in order to record 
sound and Video as taught by Morris. Also Morris discloses if the object is a 
nonvisual field object, such as a voice object, then the operator is prompted to 
input the voice record. Also, the audio check box corresponding to the audio 
record is highlighted- see Morris at Col. 4 Lines 1-15.) 
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Claim 47, Morris teaches: 

capturing a video clip of the multimedia annotation from a 
user using a video camera of the input device to annotate the paper 
document to create a multimedia paper document. 

(See Morris at Col. 2, Lines 5-20, discloses sound records, Video records, 
sequential animation sequence records, and the like, such multimedia objects 
being characterized in that their presentation requires a duration of time. It is 
within the scope of the invention that check boxes can also be used to represent 
visual fields such as other image objects, other text objects, other vector graphics 
objects, etc. Morris at Col. 3 Lines 10-15, discloses the steps that allow a user to 
add new recorded segments of audio, video, etc. for any page of data. It is 
inherently including the use of Microphone and Video camera in order to record 
sound and Video as taught by Morris.) 

Claim 48, Morris teaches: 

wherein the video camera is automatically activated when the 
user selects a reproduction function of the document reproduction 
system to reproduce the paper document. 

(See Morris at fig. 9 and Col. 9, Line 45 through Col, 10, Line 5, discloses Step 
1040 determines whether a video check box has been marked. In the example 
shown in FIG. 9A, the video check box 04 has been marked with the mark X 
which is reference 60. Step 1040 then prompts the operator to input the video 
record. Step 1042 forms a video object content architecture (VDOCA) object 58. 
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Step 1044 stores the object 58 in the MODCA envelope 40, as is shown in FIG. 
9B and adds the object name " VIDEO-1" as the pointer in the check box 04 line 
of the directory 42C (Table 5). It then adds the object name " VIDEO-1" and the 
media address " MODCA-VOCA" in the video data line of the directory 42C 
(Table 5). It is inherently including the use of Microphone and Video camera in 
order to record sound and Video as taught by Morris.) 

Claim 49, Morris teaches: 

in response to a request to retrieve a second multimedia 
document, performing a content-based search for the requested 
multimedia document within the storage based on the content of the 
multimedia annotation associated with the requested multimedia 
document. 

(See Morris at Col. 3, Lines 35-45, discloses when the check box is selected, the 
multimedia record is accessed from the MODCA envelope, for example, and is 
played. If it is an audio record, then the sound is played for the operator. If it is a 
video record, then a video cameo display window can be presented to the 
operator on the display screen and the video sequence played (i.e. played via a 
multimedia player). 

Claims 50-52, Morris teaches: 

wherein the content-based search is performed by OCR, audio 
speech recognition, or video face recognition techniques on the 
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multimedia annotations of the multimedia documents being 
searched, 

(See Morris at Col. 2, Lines 35-40, discloses scanned-in form (I.e. OCR), are 
stored in a mixed object document content architecture envelope (MODCA). 
Morris further discloses, if the box being defined is a check box, then the sub- 
type is defined as audio, video, image, text, or other type object. The resulting 
directory and the image object of the overall scanned-in form are stored in a 
mixed object document content architecture envelope (MODCA)-see Morris at 
Col. 2 Lines 20-30.) 

Claims 12, 24, and 36 are rejected under 35 U.S.C. 103(a) as being as 
being unpatentable over Morris et al US005420974A issued 05/30/1995 
(hereinafter Morris), in view of Schena et al US006448979B1 filed 01/25/1999 
(hereinafter Schena), and further in view of Hallidav et al., US 5,880,740 filed 
7/1 2/1 996 (hereinafter Halliday). 

Regarding claims 12, 24, and 36, 

Morris, and Schena do not teach, but Hallidav teaches: 

wherein the recipient receives the image of the paper 
document and the multimedia annotation in the form of Multi- 
purpose Internet Mail Extension (MIME). 

(See Halliday at Col. 8 Lines 5-28, discloses MIME.) 
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It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify Morris and Schena teaching of using a scanner capable 
of receiving data such as machine-readable code from a printed medium, so that, 
when selected by a user the barcode would accessing a segment of video file 
which references scanned document as taught by Morris, to includes MIME 
functionality as taught by Halliday. One of ordinary skill in the art would have 
been motivated to modify this combination provide a predictable result of the 
bridging the gap between the virtual multimedia-based of the Internet word and 
the physical world of printed media, wherein the annotate media may contain one 
or more of textual, audio, or video information, and the customer premises 
equipment ("CPE"), serving as the receiver, plays multimedia sequence (Schena 
col.1 , lines 5-15, col. 1 line 64 through col. 2. line 40), and provides a composite 
document as a MIME attachment to an email message and sent via an 
conventional email utility, such as the popular Eudora, America Online, or 
Netscape Navigator mail programs (see Halliday at Col. 7, Line 39, and Col. 8 
Lines 10-15). 

It is noted that any citations to specific, pages, columns, lines, or figures in 
the prior art references and any interpretation of the references should not be 
considered to be limiting in any way. A reference is relevant for all it contains 
and may be relied upon for all that it would have reasonably suggested to one 
having ordinary skill in the art. See, MPEP 2123. 
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Response to Arguments 

Applicants arguments with respect to claims 1, 4-5, 8-13, 16-17, 20-25, 
28-29, 32-36, and 44-52 have been considered but are moot in view of the new 
ground(s) of rejection. 

The Examiner introduces a new line of rejection (see above rejections for 
details). This office action is a Non-Final Rejection in order to give the applicant 
sufficient opportunity to response to the new line of rejection. 

It is noted, the examiner maintains Schena reference at this time; since 
Schena discloses the machine-readable code on the paper is used to 
communicate corresponding multimedia information when the machine-readable 
code is read by a scanner. The URL encoded in the machine-readable code 
points to a multimedia file. Thus, the machine-readable code on the printed 
medium annotates the printed medium with the referenced multimedia file, thus 
combining the printed medium with virtual multimedia. In combining the 
multimedia presentation with the printed medium, Schena has created a 
multimedia document of the claimed invention (See Schena the Abstract) and 
also Schena in col. 2 lines 6-8 that the multimedia annotation may contain one or 
more of textual, audio, or video information. Thus Schena taught some of the 
claimed invention (see the above Office Action for details). 
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Conclusion 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Quoc A. Tran whose telephone number is 
571-272-8664. The examiner can normally be reached on 9AM - 5PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Doug Hutton can be reached on 571-272-4137. The fax 
phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 

/Quoc A, Tran/ 
Patent Examiner 

Art Unit 2176 

12/12/2007 

I (Doug button] 

Doug Hutton 
Supervisory Primary Examiner 
Technology Center 2100 



